SlideShare a Scribd company logo
Vector Quantization

  Aniruddh Tyagi
     02-06-12
Voronoi Region
• Blocks:
   o   A sequence of audio.
   o   A block of image pixels.
       Formally: vector example: (0.2, 0.3, 0.5, 0.1)
• A vector quantizer maps k-dimensional vectors in the vector
  space Rk into a finite set of vectors Y = {yi: i = 1, 2, ..., N}. Each
  vector yi is called a code vector or a codeword. and the set of all
  the codewords is called a codebook. Associated with each
  codeword, yi, is a nearest neighbor region called Voronoi region,
  and it is defined by:


• The set of Voronoi regions partition the entire space Rk .
Two Dimensional Voronoi Diagram




Codewords in 2-dimensional space. Input vectors are marked with an
x, codewords are marked with red circles, and the Voronoi regions are
separated with boundary lines.
The Schematic of a Vector
Quantizer
Compression Formula
• Amount of compression:
   o Codebook size is K, input vector of dimension L
   o In order to inform the decoder of which code vector is
     selected, we need to use bits.
        E.g. need 8 bits to represent 256 code vectors.
   o Rate: each code vector contains the reconstruction value of L
     source output samples, the number of bits per sample would
     be: .
   o Sample: a scalar value in vector.
   o K: level of vector quantizer.
VQ vs SQ
Advantage of VQ over SQ:
• For a given rate, VQ results in a lower distortion than
  SQ.
• If the source output is correlate, vectors of source
  output values will tend to fall in clusters.
   o   E.g. Sayood’s book Exp 9.3.1
• Even if no dependency: greater flexibility.
   o   E.g. Sayood’s book Exp 9.3.2
Algorithms
• Lloyd algorithm: pdf-optimized
  quantizer, assume that distribution is
  known
• LBG: (VQ)
  o   Continuous (require integral ooperation)
  o   Modified: with training set.
LBG Algorithm
– Determine the number of codewords, N, or the size of the
  codebook.
– Select N codewords at random, and let that be the initial
  codebook. The initial codewords can be randomly chosen from the set
  of input vectors.
– Using the Euclidean distance measure clusterize the vectors
  around each codeword. This is done by taking each input vector and
  finding the Euclidean distance between it and each codeword. The
  input vector belongs to the cluster of the codeword that yields the
  minimum distance.
LBG Algorithm (contd.)
4.Compute the new set of codewords. This is done by obtaining the
average of each cluster. Add the component of each vector and divide by
the number of vectors in the cluster.



where i is the component of each vector (x, y, z, ... directions), m is the
number of vectors in the cluster.

5.Repeat steps 2 and 3 until the either the codewords don't change or
the change in the codewords is small.
Other Algorithms
• Problems: LBG is a greedy algorithm, may fall into
  Local minimum.
• Four methods selecting initial vectors:
  o   Random
  o   Splitting ( with perturbation vector) Animation
  o   Train with different subset
  o   PNN (pairwise nearest neighbor)
• Empty cell problem:
  o   No input correspond to a output vector
  o   Solution: give to other clusters, e.g. most populate cluster.
LBG for image compression
• Taking blocks of images as vector L=NM.
• If K vectors in code book:
   o need to use bits.
   o Rate:
• The higher the value K, the better quality, but lower compression
  ratio.
• Overhead to transmit code book:




• Train with a set of images.
Rate_Dimension Product
• Rate-dimension product
  o The size of the codebook increase exponentially with the rate.
  o Suppose we want to encode a source using R bits/sample. If
    we use an L-d quantizer, we would group L samples together
    into vectors. This means that we would have RL bits available
    to represent wach vector.
  o With RL bits, we can represent 2^(RL) output vectors.
Tree structured VQ
• Set vectors in different quadrant. Only signs of vectors need to
  be compared. Thus reduce the number of comparisons by 2^L
  for L-d vector problem.
• It works well for symmetric distribution. But not when we lose
  more and more symmetry.
Tree Structured Vector Quantizer
•   Extend to non-symmetric case:
    o   Divide the set of output points into two groups, g0 and g1, and assign to
        each group a test vector s.t. output points in each group are closer to test
        vector assigned to that group than to the test vector assigned to the other
        group.
    o   Label the two test vectors 0 and 1.
    o   When we got an input vector, compare it against the test vectors.
        Depending on the outcome, the input is compared to the output points
        associated with the test vector closest to the input.
    o   After these two comparisons, we can discard half of the output points.
    o   Comparison with the test vectors takes the place of looking at the signs of
        the components to decide which set of output points to discard from
        contention.
    o   If the total number of output points is K, we make( K/2)+2 comparisons
        instead of K comparisons.
    o   Can continue to expand the number of groups. Finally: 2logK comparisons
        instead of K.( 2 comparisons with the test vectors and a total of logK stages
Tree Structured VQ (continued)
• Since the test vectors are assigned to groups: 0, 1,
  00,01,10,11,000,001,010,011,100,101,110,111 etc.
  which are the nodes of a binary tree, the VQ has the
  name “Tree Structured VQ”.
• Penalty:
  o Possible increase in distortion: it is possible that at some
    stage the input is closer to one test vector while at the same
    time being closest to an output belonging to the rejected
    group.
  o Increase storage: output points from VQ codebook plus the
    test vectors.
Additional Links
• Slides are adapted from:
http://www.data-compression.com
and
http://www.geocities.com/mohamedqase
m/vectorquantization/vq.html

More Related Content

What's hot

Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
Universitat Politècnica de Catalunya
 
Focal loss for dense object detection
Focal loss for dense object detectionFocal loss for dense object detection
Focal loss for dense object detection
DaeHeeKim31
 
Tutorial on Object Detection (Faster R-CNN)
Tutorial on Object Detection (Faster R-CNN)Tutorial on Object Detection (Faster R-CNN)
Tutorial on Object Detection (Faster R-CNN)
Hwa Pyung Kim
 
Back propagation network
Back propagation networkBack propagation network
Back propagation network
HIRA Zaidi
 
Deep Learning for Computer Vision: Deep Networks (UPC 2016)
Deep Learning for Computer Vision: Deep Networks (UPC 2016)Deep Learning for Computer Vision: Deep Networks (UPC 2016)
Deep Learning for Computer Vision: Deep Networks (UPC 2016)
Universitat Politècnica de Catalunya
 
Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Backpropagation - Elisa Sayrol - UPC Barcelona 2018Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Convolutional Neural Networks - Veronica Vilaplana - UPC Barcelona 2018
Convolutional Neural Networks - Veronica Vilaplana - UPC Barcelona 2018Convolutional Neural Networks - Veronica Vilaplana - UPC Barcelona 2018
Convolutional Neural Networks - Veronica Vilaplana - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Implicit schemes for wave models
Implicit schemes for wave modelsImplicit schemes for wave models
Implicit schemes for wave models
Mathieu Dutour Sikiric
 
Joint unsupervised learning of deep representations and image clusters
Joint unsupervised learning of deep representations and image clustersJoint unsupervised learning of deep representations and image clusters
Joint unsupervised learning of deep representations and image clusters
Universitat Politècnica de Catalunya
 
ICIECA 2014 Paper 12
ICIECA 2014 Paper 12ICIECA 2014 Paper 12
Deep Learning for Computer Vision: Attention Models (UPC 2016)
Deep Learning for Computer Vision: Attention Models (UPC 2016)Deep Learning for Computer Vision: Attention Models (UPC 2016)
Deep Learning for Computer Vision: Attention Models (UPC 2016)
Universitat Politècnica de Catalunya
 
Crc
CrcCrc
Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...
Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...
Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...
Universitat Politècnica de Catalunya
 
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
Universitat Politècnica de Catalunya
 
Cyclic Redundancy Check
Cyclic Redundancy CheckCyclic Redundancy Check
Cyclic Redundancy Check
Rajan Shah
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
Chanuk Lim
 
Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)
Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)
Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)
Universitat Politècnica de Catalunya
 
DBSCAN (2014_11_25 06_21_12 UTC)
DBSCAN (2014_11_25 06_21_12 UTC)DBSCAN (2014_11_25 06_21_12 UTC)
DBSCAN (2014_11_25 06_21_12 UTC)
Cory Cook
 
LeNet-5
LeNet-5LeNet-5
LeNet-5
佳蓉 倪
 

What's hot (19)

Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
 
Focal loss for dense object detection
Focal loss for dense object detectionFocal loss for dense object detection
Focal loss for dense object detection
 
Tutorial on Object Detection (Faster R-CNN)
Tutorial on Object Detection (Faster R-CNN)Tutorial on Object Detection (Faster R-CNN)
Tutorial on Object Detection (Faster R-CNN)
 
Back propagation network
Back propagation networkBack propagation network
Back propagation network
 
Deep Learning for Computer Vision: Deep Networks (UPC 2016)
Deep Learning for Computer Vision: Deep Networks (UPC 2016)Deep Learning for Computer Vision: Deep Networks (UPC 2016)
Deep Learning for Computer Vision: Deep Networks (UPC 2016)
 
Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Backpropagation - Elisa Sayrol - UPC Barcelona 2018Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Backpropagation - Elisa Sayrol - UPC Barcelona 2018
 
Convolutional Neural Networks - Veronica Vilaplana - UPC Barcelona 2018
Convolutional Neural Networks - Veronica Vilaplana - UPC Barcelona 2018Convolutional Neural Networks - Veronica Vilaplana - UPC Barcelona 2018
Convolutional Neural Networks - Veronica Vilaplana - UPC Barcelona 2018
 
Implicit schemes for wave models
Implicit schemes for wave modelsImplicit schemes for wave models
Implicit schemes for wave models
 
Joint unsupervised learning of deep representations and image clusters
Joint unsupervised learning of deep representations and image clustersJoint unsupervised learning of deep representations and image clusters
Joint unsupervised learning of deep representations and image clusters
 
ICIECA 2014 Paper 12
ICIECA 2014 Paper 12ICIECA 2014 Paper 12
ICIECA 2014 Paper 12
 
Deep Learning for Computer Vision: Attention Models (UPC 2016)
Deep Learning for Computer Vision: Attention Models (UPC 2016)Deep Learning for Computer Vision: Attention Models (UPC 2016)
Deep Learning for Computer Vision: Attention Models (UPC 2016)
 
Crc
CrcCrc
Crc
 
Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...
Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...
Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...
 
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
 
Cyclic Redundancy Check
Cyclic Redundancy CheckCyclic Redundancy Check
Cyclic Redundancy Check
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
 
Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)
Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)
Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)
 
DBSCAN (2014_11_25 06_21_12 UTC)
DBSCAN (2014_11_25 06_21_12 UTC)DBSCAN (2014_11_25 06_21_12 UTC)
DBSCAN (2014_11_25 06_21_12 UTC)
 
LeNet-5
LeNet-5LeNet-5
LeNet-5
 

Viewers also liked

Basic of BISS
Basic of BISSBasic of BISS
Basic of BISS
aniruddh Tyagi
 
fundamentals_satellite_communication_part_1
fundamentals_satellite_communication_part_1fundamentals_satellite_communication_part_1
fundamentals_satellite_communication_part_1
aniruddh Tyagi
 
euler theorm
euler theormeuler theorm
euler theorm
aniruddh Tyagi
 
DIC_video_coding_standards_07
DIC_video_coding_standards_07DIC_video_coding_standards_07
DIC_video_coding_standards_07
aniruddh Tyagi
 
video_compression_2004
video_compression_2004video_compression_2004
video_compression_2004
aniruddh Tyagi
 
quantization
quantizationquantization
quantization
aniruddh Tyagi
 
Advformat_0609
Advformat_0609Advformat_0609
Advformat_0609
aniruddh Tyagi
 
Discrete cosine transform
Discrete cosine transformDiscrete cosine transform
Discrete cosine transform
aniruddh Tyagi
 
DVB_Arch
DVB_ArchDVB_Arch
DVB_Arch
aniruddh Tyagi
 
intro_dgital_TV
intro_dgital_TVintro_dgital_TV
intro_dgital_TV
aniruddh Tyagi
 
IntrRSCode
IntrRSCodeIntrRSCode
IntrRSCode
aniruddh Tyagi
 
rsa-1
rsa-1rsa-1
Basic of BISS
Basic of BISSBasic of BISS
Basic of BISS
aniruddh Tyagi
 
whitepaper_mpeg-if_understanding_mpeg4
whitepaper_mpeg-if_understanding_mpeg4whitepaper_mpeg-if_understanding_mpeg4
whitepaper_mpeg-if_understanding_mpeg4
aniruddh Tyagi
 
Discrete cosine transform
Discrete cosine transformDiscrete cosine transform
Discrete cosine transform
aniruddh Tyagi
 
fundamentals_satellite_communication_part_1
fundamentals_satellite_communication_part_1fundamentals_satellite_communication_part_1
fundamentals_satellite_communication_part_1
aniruddh Tyagi
 
euler theorm
euler theormeuler theorm
euler theorm
aniruddh Tyagi
 
DVB_Arch
DVB_ArchDVB_Arch
DVB_Arch
aniruddh Tyagi
 

Viewers also liked (18)

Basic of BISS
Basic of BISSBasic of BISS
Basic of BISS
 
fundamentals_satellite_communication_part_1
fundamentals_satellite_communication_part_1fundamentals_satellite_communication_part_1
fundamentals_satellite_communication_part_1
 
euler theorm
euler theormeuler theorm
euler theorm
 
DIC_video_coding_standards_07
DIC_video_coding_standards_07DIC_video_coding_standards_07
DIC_video_coding_standards_07
 
video_compression_2004
video_compression_2004video_compression_2004
video_compression_2004
 
quantization
quantizationquantization
quantization
 
Advformat_0609
Advformat_0609Advformat_0609
Advformat_0609
 
Discrete cosine transform
Discrete cosine transformDiscrete cosine transform
Discrete cosine transform
 
DVB_Arch
DVB_ArchDVB_Arch
DVB_Arch
 
intro_dgital_TV
intro_dgital_TVintro_dgital_TV
intro_dgital_TV
 
IntrRSCode
IntrRSCodeIntrRSCode
IntrRSCode
 
rsa-1
rsa-1rsa-1
rsa-1
 
Basic of BISS
Basic of BISSBasic of BISS
Basic of BISS
 
whitepaper_mpeg-if_understanding_mpeg4
whitepaper_mpeg-if_understanding_mpeg4whitepaper_mpeg-if_understanding_mpeg4
whitepaper_mpeg-if_understanding_mpeg4
 
Discrete cosine transform
Discrete cosine transformDiscrete cosine transform
Discrete cosine transform
 
fundamentals_satellite_communication_part_1
fundamentals_satellite_communication_part_1fundamentals_satellite_communication_part_1
fundamentals_satellite_communication_part_1
 
euler theorm
euler theormeuler theorm
euler theorm
 
DVB_Arch
DVB_ArchDVB_Arch
DVB_Arch
 

Similar to vector QUANTIZATION

Multimedia lossy compression algorithms
Multimedia lossy compression algorithmsMultimedia lossy compression algorithms
Multimedia lossy compression algorithms
Mazin Alwaaly
 
Performance of-block-codes-using-the-eigenstructure-of-the-code-correlation-m...
Performance of-block-codes-using-the-eigenstructure-of-the-code-correlation-m...Performance of-block-codes-using-the-eigenstructure-of-the-code-correlation-m...
Performance of-block-codes-using-the-eigenstructure-of-the-code-correlation-m...
Cemal Ardil
 
machine learning.pptx
machine learning.pptxmachine learning.pptx
machine learning.pptx
AbdusSadik
 
Kohonen self organizing maps
Kohonen self organizing mapsKohonen self organizing maps
Kohonen self organizing maps
raphaelkiminya
 
Fundamentals of Machine Learning.pptx
Fundamentals of Machine Learning.pptxFundamentals of Machine Learning.pptx
Fundamentals of Machine Learning.pptx
WiamFADEL
 
Sess07 Clustering02_KohonenNet.pptx
Sess07 Clustering02_KohonenNet.pptxSess07 Clustering02_KohonenNet.pptx
Sess07 Clustering02_KohonenNet.pptx
SarthakKabi2
 
DLBLR talk
DLBLR talkDLBLR talk
DLBLR talk
Anuj Gupta
 
Deep Learning Bangalore meet up
Deep Learning Bangalore meet up Deep Learning Bangalore meet up
Deep Learning Bangalore meet up
Satyam Saxena
 
Convolutional Error Control Coding
Convolutional Error Control CodingConvolutional Error Control Coding
Convolutional Error Control Coding
Mohammed Abuibaid
 
IEEE DSP Workshop 2011
IEEE DSP Workshop 2011IEEE DSP Workshop 2011
IEEE DSP Workshop 2011
Behzad Dogahe
 
IEEE DSP Workshop 2011
IEEE DSP Workshop 2011IEEE DSP Workshop 2011
IEEE DSP Workshop 2011
dogahe
 
4-RSSI-Spectral Domain Image Transforms_1.pdf
4-RSSI-Spectral Domain Image Transforms_1.pdf4-RSSI-Spectral Domain Image Transforms_1.pdf
4-RSSI-Spectral Domain Image Transforms_1.pdf
muhammadwalidmido
 
Chapter 10: Error Correction and Detection
Chapter 10: Error Correction and DetectionChapter 10: Error Correction and Detection
Chapter 10: Error Correction and Detection
JeoffnaRuth
 
Dynamic time wrapping (dtw), vector quantization(vq), linear predictive codin...
Dynamic time wrapping (dtw), vector quantization(vq), linear predictive codin...Dynamic time wrapping (dtw), vector quantization(vq), linear predictive codin...
Dynamic time wrapping (dtw), vector quantization(vq), linear predictive codin...
Tanjarul Islam Mishu
 
Deep learning book_chap_02
Deep learning book_chap_02Deep learning book_chap_02
Deep learning book_chap_02
HyeongGooKang
 
10 Backpropagation Algorithm for Neural Networks (1).pptx
10 Backpropagation Algorithm for Neural Networks (1).pptx10 Backpropagation Algorithm for Neural Networks (1).pptx
10 Backpropagation Algorithm for Neural Networks (1).pptx
SaifKhan703888
 
MDCT audio coding with pulse vector quantizers
MDCT audio coding with pulse vector quantizersMDCT audio coding with pulse vector quantizers
MDCT audio coding with pulse vector quantizers
Ericsson
 
7 convolutional codes
7 convolutional codes7 convolutional codes
7 convolutional codes
Varun Raj
 
CNN for modeling sentence
CNN for modeling sentenceCNN for modeling sentence
CNN for modeling sentence
ANISH BHANUSHALI
 
Bca1040 digital logic
Bca1040  digital logicBca1040  digital logic
Bca1040 digital logic
smumbahelp
 

Similar to vector QUANTIZATION (20)

Multimedia lossy compression algorithms
Multimedia lossy compression algorithmsMultimedia lossy compression algorithms
Multimedia lossy compression algorithms
 
Performance of-block-codes-using-the-eigenstructure-of-the-code-correlation-m...
Performance of-block-codes-using-the-eigenstructure-of-the-code-correlation-m...Performance of-block-codes-using-the-eigenstructure-of-the-code-correlation-m...
Performance of-block-codes-using-the-eigenstructure-of-the-code-correlation-m...
 
machine learning.pptx
machine learning.pptxmachine learning.pptx
machine learning.pptx
 
Kohonen self organizing maps
Kohonen self organizing mapsKohonen self organizing maps
Kohonen self organizing maps
 
Fundamentals of Machine Learning.pptx
Fundamentals of Machine Learning.pptxFundamentals of Machine Learning.pptx
Fundamentals of Machine Learning.pptx
 
Sess07 Clustering02_KohonenNet.pptx
Sess07 Clustering02_KohonenNet.pptxSess07 Clustering02_KohonenNet.pptx
Sess07 Clustering02_KohonenNet.pptx
 
DLBLR talk
DLBLR talkDLBLR talk
DLBLR talk
 
Deep Learning Bangalore meet up
Deep Learning Bangalore meet up Deep Learning Bangalore meet up
Deep Learning Bangalore meet up
 
Convolutional Error Control Coding
Convolutional Error Control CodingConvolutional Error Control Coding
Convolutional Error Control Coding
 
IEEE DSP Workshop 2011
IEEE DSP Workshop 2011IEEE DSP Workshop 2011
IEEE DSP Workshop 2011
 
IEEE DSP Workshop 2011
IEEE DSP Workshop 2011IEEE DSP Workshop 2011
IEEE DSP Workshop 2011
 
4-RSSI-Spectral Domain Image Transforms_1.pdf
4-RSSI-Spectral Domain Image Transforms_1.pdf4-RSSI-Spectral Domain Image Transforms_1.pdf
4-RSSI-Spectral Domain Image Transforms_1.pdf
 
Chapter 10: Error Correction and Detection
Chapter 10: Error Correction and DetectionChapter 10: Error Correction and Detection
Chapter 10: Error Correction and Detection
 
Dynamic time wrapping (dtw), vector quantization(vq), linear predictive codin...
Dynamic time wrapping (dtw), vector quantization(vq), linear predictive codin...Dynamic time wrapping (dtw), vector quantization(vq), linear predictive codin...
Dynamic time wrapping (dtw), vector quantization(vq), linear predictive codin...
 
Deep learning book_chap_02
Deep learning book_chap_02Deep learning book_chap_02
Deep learning book_chap_02
 
10 Backpropagation Algorithm for Neural Networks (1).pptx
10 Backpropagation Algorithm for Neural Networks (1).pptx10 Backpropagation Algorithm for Neural Networks (1).pptx
10 Backpropagation Algorithm for Neural Networks (1).pptx
 
MDCT audio coding with pulse vector quantizers
MDCT audio coding with pulse vector quantizersMDCT audio coding with pulse vector quantizers
MDCT audio coding with pulse vector quantizers
 
7 convolutional codes
7 convolutional codes7 convolutional codes
7 convolutional codes
 
CNN for modeling sentence
CNN for modeling sentenceCNN for modeling sentence
CNN for modeling sentence
 
Bca1040 digital logic
Bca1040  digital logicBca1040  digital logic
Bca1040 digital logic
 

More from aniruddh Tyagi

whitepaper_mpeg-if_understanding_mpeg4
whitepaper_mpeg-if_understanding_mpeg4whitepaper_mpeg-if_understanding_mpeg4
whitepaper_mpeg-if_understanding_mpeg4
aniruddh Tyagi
 
BUC BLOCK UP CONVERTER
BUC BLOCK UP CONVERTERBUC BLOCK UP CONVERTER
BUC BLOCK UP CONVERTER
aniruddh Tyagi
 
digital_set_top_box2
digital_set_top_box2digital_set_top_box2
digital_set_top_box2
aniruddh Tyagi
 
DCT
DCTDCT
EBU_DVB_S2 READY TO LIFT OFF
EBU_DVB_S2 READY TO LIFT OFFEBU_DVB_S2 READY TO LIFT OFF
EBU_DVB_S2 READY TO LIFT OFF
aniruddh Tyagi
 
ADVANCED DVB-C,DVB-S STB DEMOD
ADVANCED DVB-C,DVB-S STB DEMODADVANCED DVB-C,DVB-S STB DEMOD
ADVANCED DVB-C,DVB-S STB DEMOD
aniruddh Tyagi
 
haffman coding DCT transform
haffman coding DCT transformhaffman coding DCT transform
haffman coding DCT transform
aniruddh Tyagi
 
Classification
ClassificationClassification
Classification
aniruddh Tyagi
 
tyagi 's doc
tyagi 's doctyagi 's doc
tyagi 's doc
aniruddh Tyagi
 
quantization_PCM
quantization_PCMquantization_PCM
quantization_PCM
aniruddh Tyagi
 
ECMG & EMMG protocol
ECMG & EMMG protocolECMG & EMMG protocol
ECMG & EMMG protocol
aniruddh Tyagi
 
7015567A
7015567A7015567A
7015567A
aniruddh Tyagi
 
art_sklar7_reed-solomon
art_sklar7_reed-solomonart_sklar7_reed-solomon
art_sklar7_reed-solomon
aniruddh Tyagi
 
DVBSimulcrypt2
DVBSimulcrypt2DVBSimulcrypt2
DVBSimulcrypt2
aniruddh Tyagi
 
en_302769v010101v
en_302769v010101ven_302769v010101v
en_302769v010101v
aniruddh Tyagi
 
Euler formula
Euler formulaEuler formula
Euler formula
aniruddh Tyagi
 
RSA
RSARSA
video compression
video compressionvideo compression
video compression
aniruddh Tyagi
 
fundamentals of linux
fundamentals of linuxfundamentals of linux
fundamentals of linux
aniruddh Tyagi
 
IBCBarconetTransratingEfficiency
IBCBarconetTransratingEfficiencyIBCBarconetTransratingEfficiency
IBCBarconetTransratingEfficiency
aniruddh Tyagi
 

More from aniruddh Tyagi (20)

whitepaper_mpeg-if_understanding_mpeg4
whitepaper_mpeg-if_understanding_mpeg4whitepaper_mpeg-if_understanding_mpeg4
whitepaper_mpeg-if_understanding_mpeg4
 
BUC BLOCK UP CONVERTER
BUC BLOCK UP CONVERTERBUC BLOCK UP CONVERTER
BUC BLOCK UP CONVERTER
 
digital_set_top_box2
digital_set_top_box2digital_set_top_box2
digital_set_top_box2
 
DCT
DCTDCT
DCT
 
EBU_DVB_S2 READY TO LIFT OFF
EBU_DVB_S2 READY TO LIFT OFFEBU_DVB_S2 READY TO LIFT OFF
EBU_DVB_S2 READY TO LIFT OFF
 
ADVANCED DVB-C,DVB-S STB DEMOD
ADVANCED DVB-C,DVB-S STB DEMODADVANCED DVB-C,DVB-S STB DEMOD
ADVANCED DVB-C,DVB-S STB DEMOD
 
haffman coding DCT transform
haffman coding DCT transformhaffman coding DCT transform
haffman coding DCT transform
 
Classification
ClassificationClassification
Classification
 
tyagi 's doc
tyagi 's doctyagi 's doc
tyagi 's doc
 
quantization_PCM
quantization_PCMquantization_PCM
quantization_PCM
 
ECMG & EMMG protocol
ECMG & EMMG protocolECMG & EMMG protocol
ECMG & EMMG protocol
 
7015567A
7015567A7015567A
7015567A
 
art_sklar7_reed-solomon
art_sklar7_reed-solomonart_sklar7_reed-solomon
art_sklar7_reed-solomon
 
DVBSimulcrypt2
DVBSimulcrypt2DVBSimulcrypt2
DVBSimulcrypt2
 
en_302769v010101v
en_302769v010101ven_302769v010101v
en_302769v010101v
 
Euler formula
Euler formulaEuler formula
Euler formula
 
RSA
RSARSA
RSA
 
video compression
video compressionvideo compression
video compression
 
fundamentals of linux
fundamentals of linuxfundamentals of linux
fundamentals of linux
 
IBCBarconetTransratingEfficiency
IBCBarconetTransratingEfficiencyIBCBarconetTransratingEfficiency
IBCBarconetTransratingEfficiency
 

Recently uploaded

UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
David Brossard
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 

Recently uploaded (20)

UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 

vector QUANTIZATION

  • 1. Vector Quantization Aniruddh Tyagi 02-06-12
  • 2. Voronoi Region • Blocks: o A sequence of audio. o A block of image pixels. Formally: vector example: (0.2, 0.3, 0.5, 0.1) • A vector quantizer maps k-dimensional vectors in the vector space Rk into a finite set of vectors Y = {yi: i = 1, 2, ..., N}. Each vector yi is called a code vector or a codeword. and the set of all the codewords is called a codebook. Associated with each codeword, yi, is a nearest neighbor region called Voronoi region, and it is defined by: • The set of Voronoi regions partition the entire space Rk .
  • 3. Two Dimensional Voronoi Diagram Codewords in 2-dimensional space. Input vectors are marked with an x, codewords are marked with red circles, and the Voronoi regions are separated with boundary lines.
  • 4. The Schematic of a Vector Quantizer
  • 5. Compression Formula • Amount of compression: o Codebook size is K, input vector of dimension L o In order to inform the decoder of which code vector is selected, we need to use bits.  E.g. need 8 bits to represent 256 code vectors. o Rate: each code vector contains the reconstruction value of L source output samples, the number of bits per sample would be: . o Sample: a scalar value in vector. o K: level of vector quantizer.
  • 6. VQ vs SQ Advantage of VQ over SQ: • For a given rate, VQ results in a lower distortion than SQ. • If the source output is correlate, vectors of source output values will tend to fall in clusters. o E.g. Sayood’s book Exp 9.3.1 • Even if no dependency: greater flexibility. o E.g. Sayood’s book Exp 9.3.2
  • 7. Algorithms • Lloyd algorithm: pdf-optimized quantizer, assume that distribution is known • LBG: (VQ) o Continuous (require integral ooperation) o Modified: with training set.
  • 8. LBG Algorithm – Determine the number of codewords, N, or the size of the codebook. – Select N codewords at random, and let that be the initial codebook. The initial codewords can be randomly chosen from the set of input vectors. – Using the Euclidean distance measure clusterize the vectors around each codeword. This is done by taking each input vector and finding the Euclidean distance between it and each codeword. The input vector belongs to the cluster of the codeword that yields the minimum distance.
  • 9. LBG Algorithm (contd.) 4.Compute the new set of codewords. This is done by obtaining the average of each cluster. Add the component of each vector and divide by the number of vectors in the cluster. where i is the component of each vector (x, y, z, ... directions), m is the number of vectors in the cluster. 5.Repeat steps 2 and 3 until the either the codewords don't change or the change in the codewords is small.
  • 10. Other Algorithms • Problems: LBG is a greedy algorithm, may fall into Local minimum. • Four methods selecting initial vectors: o Random o Splitting ( with perturbation vector) Animation o Train with different subset o PNN (pairwise nearest neighbor) • Empty cell problem: o No input correspond to a output vector o Solution: give to other clusters, e.g. most populate cluster.
  • 11. LBG for image compression • Taking blocks of images as vector L=NM. • If K vectors in code book: o need to use bits. o Rate: • The higher the value K, the better quality, but lower compression ratio. • Overhead to transmit code book: • Train with a set of images.
  • 12. Rate_Dimension Product • Rate-dimension product o The size of the codebook increase exponentially with the rate. o Suppose we want to encode a source using R bits/sample. If we use an L-d quantizer, we would group L samples together into vectors. This means that we would have RL bits available to represent wach vector. o With RL bits, we can represent 2^(RL) output vectors.
  • 13. Tree structured VQ • Set vectors in different quadrant. Only signs of vectors need to be compared. Thus reduce the number of comparisons by 2^L for L-d vector problem. • It works well for symmetric distribution. But not when we lose more and more symmetry.
  • 14. Tree Structured Vector Quantizer • Extend to non-symmetric case: o Divide the set of output points into two groups, g0 and g1, and assign to each group a test vector s.t. output points in each group are closer to test vector assigned to that group than to the test vector assigned to the other group. o Label the two test vectors 0 and 1. o When we got an input vector, compare it against the test vectors. Depending on the outcome, the input is compared to the output points associated with the test vector closest to the input. o After these two comparisons, we can discard half of the output points. o Comparison with the test vectors takes the place of looking at the signs of the components to decide which set of output points to discard from contention. o If the total number of output points is K, we make( K/2)+2 comparisons instead of K comparisons. o Can continue to expand the number of groups. Finally: 2logK comparisons instead of K.( 2 comparisons with the test vectors and a total of logK stages
  • 15. Tree Structured VQ (continued) • Since the test vectors are assigned to groups: 0, 1, 00,01,10,11,000,001,010,011,100,101,110,111 etc. which are the nodes of a binary tree, the VQ has the name “Tree Structured VQ”. • Penalty: o Possible increase in distortion: it is possible that at some stage the input is closer to one test vector while at the same time being closest to an output belonging to the rejected group. o Increase storage: output points from VQ codebook plus the test vectors.
  • 16. Additional Links • Slides are adapted from: http://www.data-compression.com and http://www.geocities.com/mohamedqase m/vectorquantization/vq.html